Image diagnosis assisting apparatus, image diagnosis assisting system and image diagnosis assisting method

ABSTRACT

An image diagnosis assisting apparatus according to the present invention executes: processing of inputting an image of a tissue or cell; processing of extracting a feature amount of a tissue or cell from a processing target image; processing of extracting a feature amount of a tissue or cell from an image having a component different from that of the target image; and processing of determining presence or absence of a lesion and lesion probability for each of the target images by using a plurality of the feature amounts.

TECHNICAL FIELD

The present invention relates to an image diagnosis assisting apparatus,an image diagnosis assisting system, and an image diagnosis assistingmethod, and to an image processing technology for detecting specifictissues or cells (for example, cancer) included in an image of, forexample, a slice of tissues or cells on a slide glass captured by animage capturing apparatus such as a camera mounted on a microscope, forexample.

BACKGROUND ART

In recent years, in the diagnosis of illness, “pathological diagnosis”using microscopic observation of tissue preparation of a lesioned partoccupies a significant position. In the pathological diagnosis, theprocess from specimen preparation to diagnosis requires a lot ofmanpower, and it is difficult to automate the process. In particular,ability and experiment of a pathologist are important in diagnosis, andthe diagnosis depends on the personal ability of the pathologist.Meanwhile, since the number of cancer patients increases due topopulation aging, for example, there is a shortage of pathologists atmedical sites. From the above, there is an increasing need for an imageprocessing technology or remote diagnosis that supports the pathologicaldiagnosis.

In order to determine whether tissues are pathological tissues or not tosupport the pathological diagnosis in this way, for example, there is atechnology proposed in Patent Document 1. In Patent Document 1, alow-magnification image is generated from a high-magnification image, asimple image classification is made with the low-magnification image,and then, pathological tissues are classified with the use of thehigh-magnification image, from which the low-magnification image hasbeen generated.

PRIOR ART DOCUMENT Patent Document

-   Patent Document 1: JP-2010-203949-A

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

With regard to tissue or cell images, however, whether there is anabnormal tissue (for example, cancer) or an abnormal cell (for example,cancer) or not cannot be determined only from a tissue or cell imagestained by one kind of staining method, resulting in detection failureor false detection, which is a problem. Thus, even when alow-magnification image is generated from a high-magnification image, asimple image classification is made with the low-magnification image,and then, tissues or cells are classified with the use of thehigh-magnification image, from which the low-magnification image hasbeen generated as in Patent Document 1, abnormal tissues or abnormalcells cannot be detected only from an tissue or cell image stained byone kind of staining method, resulting in detection failure or falsedetection, which is a problem. Further, in a case where an image iscreated by a plurality of staining methods, an inspection cost is high,which is a problem.

The present invention has been made in view of such circumstances, andprovides a technology for implementing a tissue or cell classificationfrom a tissue or cell image stained by one kind of staining method, bynot only calculating a feature amount of the stained tissue or cellimage, but also estimating a feature amount of a tissue or cell imagestained by another staining method, from the tissue or cell imagestained by one kind of staining method.

Means for Solving the Problems

In order to solve the above-mentioned problems, the present inventionincludes: a processor configured to execute various programs forperforming image processing on a target image; and a memory configuredto store a result of the image processing, in which the processorexecutes: processing of inputting an image of a tissue or cell;processing of extracting a feature amount of a tissue or cell in thetarget image; feature extraction processing of extracting a featureamount of a tissue or cell in an image having a component different froma component of the target image; and determination processing ofdetermining presence or absence of a lesion and lesion probability foreach of the target images by using a plurality of the feature amounts.

Further, the present invention includes: a processor configured toexecute various programs for performing image processing on a targetimage; and a memory configured to store a result of the imageprocessing, in which the processor executes: processing of inputting animage of a tissue or cell; processing of extracting a feature amount ofa tissue or cell in the target image; processing of generating, from thetarget image, an image having a component different from a component ofthe target image; feature extraction processing of extracting a featureamount of a tissue or cell in the generated image; and determinationprocessing of determining presence or absence of a lesion and lesionprobability for each of the target images by using a plurality of thefeature amounts.

More features related to the present invention will be apparent from thedescription and the attached drawings of the present specification.Further, aspects of the present invention are achieved and implementedby elements, various combinations of the elements, the followingdetailed description, and aspects of the appended claims. Thedescription of the present specification is merely a typical example,and it should be understood that the description does not limit theclaims of the present invention or application examples thereof to anymeaning.

Effects of the Invention

According to the present invention, even in a case where a tissue orcell image stained by a plurality of kinds of staining methods isnecessary to determine whether tissues or cells are abnormal or not, thetissues or cells can be classified from a tissue or cell image stainedby one kind of staining method, by not only calculating a feature amountof the stained tissue or cell image, but also estimating a featureamount of a tissue or cell image stained by another staining method,from the tissue or cell image stained by one kind of staining method, tothereby prevent false detection or over-detection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating the functions of an imagediagnosis assisting apparatus according to a first embodiment of thepresent invention.

FIG. 2 is a diagram illustrating an example of the hardwareconfiguration of an image diagnosis assisting apparatus according tofirst and second embodiments of the present invention.

FIG. 3 is a diagram illustrating an example of the operation of afeature extracting unit.

FIG. 4 is a diagram illustrating an example of the operation of thefeature extracting unit.

FIG. 5 is a diagram illustrating an example of the operation of aone-classification determining unit.

FIG. 6 is a diagram illustrating an example of the operation of alearning unit.

FIG. 7 is a diagram illustrating an example of the operation of thelearning unit.

FIG. 8 is a diagram illustrating an example of the operation of adrawing unit.

FIG. 9 is a flow chart illustrating the operation of the learning unit.

FIG. 10 is a flow chart illustrating the whole operation of the imagediagnosis assisting apparatus according to the first embodiment.

FIGS. 11A, 11B, and 11C are diagrams illustrating an example ofdetermination result display by the drawing unit.

FIG. 12 is a block diagram illustrating the functions of the imagediagnosis assisting apparatus according to the second embodiment of thepresent invention.

FIG. 13 is a diagram illustrating an example of the operation of animage generating unit.

FIG. 14 is a flow chart illustrating the whole operation of the imagediagnosis assisting apparatus according to the second embodiment.

FIG. 15 is a diagram illustrating the schematic configuration of aremote diagnosis assisting system having mounted thereon the imagediagnosis assisting apparatus of the present invention.

FIG. 16 is a diagram illustrating the schematic configuration of anonline contract service providing system having mounted thereon theimage diagnosis assisting apparatus of the present invention.

MODES FOR CARRYING OUT THE INVENTION

Embodiments of the present invention provide an image diagnosisassisting apparatus configured to, from a tissue or cell image stainedby one kind of staining method, calculate a feature amount of thestained tissue or cell image, and estimate a feature amount of a tissueor cell image stained by another staining method, to thereby preventdetection failure or false detection of abnormal tissues or abnormalcells (for example, lesion), and a method therefor.

Now, the embodiments of the present invention are described withreference to the attached drawings. In the attached drawings, the samefunctional elements are sometimes denoted by the same numbers. Notethat, the attached drawings illustrate specific embodiments andimplementation examples in accordance with the principle of the presentinvention, but the drawings are intended to facilitate an understandingof the present invention and are by no means used for limiting theinterpretation of the present invention.

In the present embodiments, the embodiments are described in detailenough for those skilled in the art to implement the present invention,but other implementation forms and modes are also possible. It should beunderstood that changes of configurations or structures or replacementof various elements are possible without departing from the range andspirit of the technical idea of the present invention. The followingdescription should therefore not be interpreted as being limitedthereto.

Furthermore, as described later, the embodiments of the presentinvention may be implemented by software that runs on a general-purposecomputer, or may be implemented by dedicated hardware or by acombination of the software and the hardware.

In the following, each processing in the embodiments of the presentinvention is described by regarding “each processing unit (for example,feature extracting unit) that functions as a program” as a subject(operation subject). The program, however, performs processingdetermined by a processor (CPU or the like) executing the program, whileusing a memory and a communication port (communication controlapparatus), and thus, the processor may be regarded as the subject inthe description.

(1) First Embodiment

<Functional Configuration of Image Diagnosis Assisting Apparatus>

FIG. 1 is a block diagram illustrating the functional configuration ofan image diagnosis assisting apparatus according to the embodiment ofthe present invention.

An image diagnosis assisting apparatus 1 includes an input unit 10, afeature extracting unit 11, a one-classification determining unit 12, adrawing unit 13, a recording unit 14, a learning unit 15, a control unit91, and a memory 90. The image diagnosis assisting apparatus may bemounted in a tissue or cell image acquiring apparatus, such as a virtualslide apparatus, or may be mounted in a server that is connected to thetissue or cell image acquiring apparatus via a network as describedlater (third and fourth embodiments).

In the image diagnosis assisting apparatus 1, the input unit 10, thefeature extracting unit 11, the one-classification determining unit 12,the drawing unit 13, the recording unit 14, and the learning unit 15 maybe implemented by programs or may be implemented by modularization.

Image data is input to the input unit 10. For example, the input unit 10may acquire still image data or such data taken at a predetermined timeinterval by imaging means, such as a camera built in a microscope, to beencoded in JPG, Jpeg 2000, PNG, or the BMP format, for example. Theinput unit 10 may use the image as an input image. Further, the inputunit 10 may extract still image data of frames at a predeterminedinterval from moving image data in, for example, Motion JPEG, MPEG,H.264, or the HD/SDI format, and may use the image as an input image.Further, the input unit 10 may use, as an input image, an image acquiredby the imaging means via a bus or the network. Further, the input unit10 may use, as an input image, an image already stored in an attachableand detachable storage medium.

The feature extracting unit 11 calculates, from a tissue or cell imagestained by one kind of staining method, a feature amount of tissues orcells in the stained tissue or cell image, and estimates a featureamount of tissues or cells in a tissue or cell image stained by anotherstaining method.

The one-classification determining unit 12 calculates abnormalprobability of tissues or cells from an extracted feature amount and anestimated feature amount, and classifies whether an input image includesnormal tissues, abnormal tissues, normal cells, or abnormal cells.

The drawing unit 13 draws a detection frame on an image to surroundabnormal tissues or abnormal cells classified by the one-classificationdetermining unit 12.

The recording unit 14 saves, in the memory 90, an image obtained by thedrawing unit 13 drawing a detection frame on an original image.

The learning unit 15 calculates each parameter (filter factor, offsetvalue, or other matters) necessary for discrimination by machinelearning so that normal tissues or cells in an input image arediscriminated as normal tissues or cells and abnormal tissues or cellsin the input image are discriminated as abnormal tissues or cells. Inaddition, the learning unit 15 calculates each parameter (filter factor,offset value, or other matters) necessary for estimation by machinelearning so that, from the input image, normal tissues or cells in atissue or cell image stained by another staining method, which isdifferent from a staining method for the input image, are estimated asnormal tissues or cells, and abnormal tissues or cells in the tissue orcell image stained by another staining method, which is different fromthe staining method for the input image, are estimated as abnormaltissues or cells.

The control unit 91 is implemented by a processor and is connected toeach element in the image diagnosis assisting apparatus 1. Each elementof the image diagnosis assisting apparatus 1 operates by the autonomousoperation of each components described above or by instructions from thecontrol unit 91.

Thus, in the image diagnosis assisting apparatus 1 of the presentembodiment, the one-classification determining unit 12 classifieswhether an input image includes normal tissues, abnormal tissues, normalcells, or abnormal cells by using a feature amount indicating theabnormal probability of tissues or cells in the input image and afeature amount indicating the abnormal probability of tissues or cellsin an image stained by another staining method different from a stainingmethod for the input image, these feature amount obtained by the featureextracting unit 11.

<Hardware Configuration of Image Diagnosis Assisting Apparatus>

FIG. 2 is a diagram illustrating an example of the hardwareconfiguration of the image diagnosis assisting apparatus 1 according tothe embodiment of the present invention.

The image diagnosis assisting apparatus 1 includes a CPU (processor) 201configured to execute various programs, a memory 202 configured to storevarious programs, a storage apparatus 203 (corresponding to memory 90)configured to store various pieces of data, an output apparatus 204configured to output after-detection images, an input apparatus 205configured to receive, for example, instructions from a user or images,and a communication device 206 configured to establish communicationwith another apparatus. These components are connected to each other bya bus 207.

The CPU 201 reads various programs from the memory 202 as needed toexecute the programs.

The memory 202 stores, as the programs, the input unit 10, the featureextracting unit 11, the one-classification determining unit 12, thedrawing unit 13, the recording unit 14, and the learning unit 15. Notethat, the memory 202 of the image diagnosis assisting apparatus 1according to the first embodiment does not include an image generatingunit 20.

The storage apparatus 203 stores, for example, processing target images,a classification result of an input image generated by theone-classification determining unit 12 and a numerical value thereof, anestimation result of an image stained by another staining methoddifferent from a staining method for the input image and a numericalvalue thereof, positional information for drawing a detection framegenerated by the drawing unit 13, and each parameter of Expression (1)and Expression (2) generated by the learning unit 15. Expression (1) andExpression (2) are described later.

The output apparatus 204 includes a device, such as a display, aprinter, or a speaker. For example, the output device 204 displays datagenerated by the drawing unit 13 on a display screen.

The input apparatus 205 includes a device, such as a keyboard, a mouse,or a microphone. The image diagnosis assisting apparatus 1 receives, bythe input apparatus 205, an instruction by the user (includingdetermination of a processing target image).

The communication device 206 is not necessarily provided to the imagediagnosis assisting apparatus 1. In a case where a personal computer orthe like connected to the tissue or cell image acquiring apparatusincludes a communication device, the image diagnosis assisting apparatus1 may not include the communication device 206. For example, thecommunication device 206 performs an operation of receiving data(including image) sent from another apparatus (for example, server)connected thereto via the network, thereby storing the data in thestorage apparatus 203.

The image diagnosis assisting apparatus of the present inventioncalculates a feature amount of tissues or cells in an input image, andestimates, from the input image, a feature amount of tissues or cells inan image stained by another staining method different from a stainingmethod for the input image, to thereby determine the lesion probabilityof the tissues or cells in the input image by using these featureamounts.

<Configuration and Operation of Each Unit>

Now, the configuration and operation of each element are described indetail.

(i) Feature Extracting Unit 11

The feature extracting unit 11 obtains feature amounts of an input imageand an image stained by another staining method different from astaining method for the input image. As an example, how each featureamount is obtained is illustrated in FIG. 3 . CNN in FIG. 3 indicates aconvolutional neural network.

For example, with Expression (1), the feature extracting unit 11 obtainsa feature amount FAi of tissues or cells in an input image A1 from theinput image A1 by using a feature extractor A. Further, with Expression(1), the feature extracting unit 11 obtains, from the input image A1, afeature amount FCi of tissues or cells in an image having a componentdifferent from that of the input image by using a feature extractor C.

A filter factor wj in Expression (1) is a factor obtained by, forexample, machine learning so that normal tissues or normal cells arediscriminated as normal tissues or normal cells and abnormal tissues orabnormal cells are discriminated as abnormal tissues or abnormal cells.

In Expression (1), pi indicates a pixel value, bi indicates an offsetvalue, m indicates a value of a filter factor, and h indicates anonlinear function. As illustrated in FIG. 4 , with the use ofExpression (1), a calculation result of each filter with respect to atarget image is obtained from the upper left to the lower right toobtain a feature amount fi of a given filter i. For example, the matrixof the feature amount fi obtained by the feature extractor A is regardedas the feature amount FAi of the input image A1. In a similar manner,the matrix of the feature amount fi obtained by the feature extractor Cis regarded as the feature amount FCi estimated from the input image A1.A creation method for the feature extractors A and C is described laterin association with the learning unit 15.fi=h(Σ_(j=1) ^(m)(pj×wj)+bi)[Math. 1](ii) One-Classification Determining Unit 12

The one-classification determining unit 12 uses, as illustrated in FIG.5 , a matrix f of the feature amount FAi of the feature extractor A andthe feature amount FCi of the feature extractor C, which have beenobtained by the feature extracting unit 11, to calculate a value oflesion probability by logistic regression with Expression (2), tothereby determine whether tissues or cells in the input image A1 arenormal or abnormal. In Expression (2), w indicates the matrix of aweight, b indicates an offset value, g indicates a nonlinear function,and y indicates a calculation result. The learning unit 15, which isdescribed later, obtains the weight w and the offset value b by machinelearning.y=g(w×f+b)  [Math. 2]

As an example, even in a case where the presence or absence of a lesioncannot be determined from an HE stained image of a prostate, with theuse of the feature extractors A and C, the feature amount FAi iscalculated from the HE stained image of the prostate and the featureamount FCi is calculated from the HE stained image of the prostate toestimate a feature amount of the immunostained image of the prostate,thereby clarifying a feature regarding the presence or absence of basalcells or the presence or absence of a lesion on epithelial cells. Inthis way, the presence or absence of a lesion that cannot be determinedfrom an HE stained image alone can be determined.

(iii) Learning Unit 15

The learning unit 15 learns a feature amount of tissues or cells in aninput tissue or cell image by using, for example, the machine learningtechnology, which is the related art, so that when the tissues or cellsare normal tissues or normal cells, the tissues or cells are determinedas normal tissues or normal cells by logistic regression with Expression(2), for example. Further, the learning unit 15 learns a feature amountof tissues or cells in an input tissue or cell image so that when thetissues or cells are abnormal tissues or abnormal cells, the tissues orcells are determined as abnormal tissues or abnormal cells by logisticregression. As the machine learning technology, for example, aconvolutional neural network may be used.

As illustrated in FIG. 6 , through prior machine learning, the learningunit 15 uses the input image A1 (for example, HE stained image) and animage B1 having a component different from that of the input image (forexample, immunostained image or image subjected to special stains) tocreate, with Expression (1) and Expression (2), the feature extractor Aconfigured to calculate the feature amount fi of the input image A1(denoted by FAi) and a feature extractor B configured to calculate thefeature amount fi of the image B1 having a component different from thatof the input image (denoted by FBi) so that abnormal tissues or abnormalcells are determined as abnormal tissues or abnormal cells, and normaltissues or normal cells are determined as normal tissues or normalcells.

As illustrated in FIG. 7 , through prior machine learning, the learningunit 15 further uses the feature extractor A and the feature extractor Bto create, with Expression (1) and Expression (2), a feature extractor Cthat achieves a small difference between the feature amount FBi that iscalculated when the image B1 having a component different from that ofthe input image is input to the feature extractor B and the featureamount fi (denoted by FCi) that is calculated when the input image A1 isinput to the feature extractor C. With the feature extractor C createdin this way, from the input image A1, the feature amount FCi of theimage having a component different from that of the input image can beestimated.

The learning unit 15 uses, with the feature extracting unit 11 and theone-classification determining unit 12 repeatedly performing theprocessing, a plurality of images for learning to obtain the weight w,the filter factor wj, and the offset values b and bi in Expression (1)and Expression (2), thereby creating the feature extractor A configuredto calculate the feature amount FAi of the input image A1 from the inputimage A1 and the feature extractor C configured to calculate, from theinput image A1, the feature amount FCi of the image having a componentdifferent from that of the input image.

The learning unit 15 obtains the weight w, the filter factor wj, and theoffset values b and bi for each of a case where a matrix including thefeature amount FAi and the feature amount FCi is regarded as f ((a)), acase where a matrix only including the feature amount FAi is regarded asf ((b)), and a case where a matrix only including the feature amount FCiis regarded as f ((c)). The learning unit 15 stores, in the memory, theweights w, the filter factors wj, and the offset values b and bi, whichhave been obtained.

(iv) Drawing Unit 13

The drawing unit 13 draws, in a case where the one-classificationdetermining unit 12 has determined tissues or cells as abnormal, adetection frame on an input target image to indicate locations ofsuspicious abnormal tissues or abnormal cells as illustrated in FIG. 8 .

Meanwhile, the drawing unit 13 draws no detection frame on the inputtarget image and displays the input target image as it is in a casewhere the tissues or cells have been determined as normal. Further, asillustrated in FIG. 8 , the drawing unit 13 displays a result ofdetermined lesion probability (for example, tumor). Further, as anexample, the drawing unit 13 displays a result of lesion probabilitydetermination in a graphical user interface (GUI) illustrated in FIG. 11.

FIG. 11 is a diagram of an example of a case of stomach cancer, andillustrates a classification result of poorly differentiated tubularadenocarcinoma, moderately differentiated tubular adenocarcinoma, welldifferentiated tubular adenocarcinoma, papillary adenocarcinoma, andsignet ring cell carcinoma. In the example of FIG. 11 , with regard topoorly differentiated tubular adenocarcinoma, the one-classificationdetermining unit 12 makes a classification that an input target imageincludes poorly differentiated tubular adenocarcinoma, which correspondsto abnormal tissues or cells, and calculates a value of lesionprobability (HE) of the poorly differentiated tubular adenocarcinoma as0.69 and a value of lesion probability (immunohistochemistry/specialstains) thereof as 0.80.

Further, with regard to moderately differentiated tubularadenocarcinoma, the one-classification determining unit 12 makes aclassification that the input target image does not include moderatelydifferentiated tubular adenocarcinoma, which corresponds to abnormaltissues or cells, and only includes normal tissues or cells, andcalculates a value of lesion probability (HE) of the moderatelydifferentiated tubular adenocarcinoma as 0.11 and a value of lesionprobability (immunohistochemistry/special stains) thereof as 0.10.

Further, with regard to well differentiated tubular adenocarcinoma, theone-classification determining unit 12 makes a classification that theinput target image does not include well differentiated tubularadenocarcinoma, which corresponds to abnormal tissues or cells, and onlyincludes normal tissues or cells, and calculates a value of lesionprobability (HE) of the well differentiated tubular adenocarcinoma as0.09 and a value of lesion probability (immunohistochemistry/specialstains) thereof as 0.05.

Further, with regard to papillary adenocarcinoma, the one-classificationdetermining unit 12 makes a classification that the input target imagedoes not include papillary adenocarcinoma, which corresponds to abnormaltissues or cells, and only includes normal tissues or cells, andcalculates a value of lesion probability (HE) of the papillaryadenocarcinoma as 0.06 and a value of lesion probability(immunohistochemistry/special stains) thereof as 0.03.

Further, with regard to signet ring cell carcinoma, theone-classification determining unit 12 makes a classification that theinput target image does not include signet ring cell carcinoma, whichcorresponds to abnormal tissues or cells, and only includes normaltissues or cells, and calculates a value of lesion probability (HE) ofthe signet ring cell carcinoma as 0.05 and a value of lesion probability(immunohistochemistry/special stains) thereof as 0.02.

(v) Recording Unit 14

The recording unit 14 saves, in the memory 90, coordinate informationwith which the drawing unit 13 draws a detection frame on a target imageinput to the drawing unit 13, and the target image.

<Processing Procedure of Image Diagnosis Assisting Apparatus>

FIG. 9 is a flow chart illustrating the operation of the learning unit15 of the image diagnosis assisting apparatus 1 according to theembodiment of the present invention. In the following description, thelearning unit 15 is regarded as an operation subject, but thedescription may be read as having, as the operation subject, the CPU 201configured to execute each processing unit serving as the program.

(i) Step 901

The input unit 10 receives an image for learning input thereto, andoutputs the input image to the learning unit 15.

(ii) Step 902

Through machine learning, the learning unit 15 uses the filters toobtain, with Expression (1) and Expression (2) described above, thefeature amount FAi of the tissues or cells in the input image A1 and thefeature amount FBi of the image B1 having a component different fromthat of the input image, to thereby create the feature extractors A andB.

(iii) Step 903

Through machine learning, the learning unit 15 uses the featureextractors A and B and the filter to create, with Expression (1) andExpression (2), the feature extractor C that achieves a small differencebetween the feature amount FBi that is calculated when the image B1having a component different from that of the input image is input tothe feature extractor B and the feature amount fi (denoted by FCi) thatis calculated when the input image A1 is input to the feature extractorC.

The weight w and the offset values b of logistic regression layer, andthe filter factor wj and the offset values bi are obtained for each ofthe cases where a matrix including the feature amount FAi and thefeature amount FCi is regarded as f ((a)), the case where a matrix onlyincluding the feature amount FAi is regarded as f ((b)), and the casewhere a matrix only including the feature amount FCi is regarded as f((c)).

(iv) Step 904

The learning unit 15 saves, in the memory 90, the weight w, the filterfactor wj, and the offset values b and bi, which have been calculated,of each of the feature extractors A and C.

FIG. 10 is a flow chart illustrating the operation of the imagediagnosis assisting apparatus 1 of the present embodiment. In thefollowing description, each processing unit (input unit 10, featureextracting unit 11, or another unit) is regarded as an operationsubject, but the description may be read as having, as the operationsubject, the CPU 201 configured to execute each processing unit servingas the program.

(i) Step S1001

The input unit 10 outputs the input image A1 to the feature extractingunit 11.

(ii) Step S1002

The feature extracting unit 11 reads the filter factor wj and the offsetbi of each of the feature extractors A and C from the memory 90. Then,with Expression (1) described above, the feature extracting unit 11obtains, with the use of the filters, the feature amount FAi of thetissues or cells in the input image A1 and the feature amount FCi oftissues or cells estimated from the input image A1.

(iii) Step S1003

The one-classification determining unit 12 reads the weight w and theoffset b of each of logistic regression layer using the feature amountFAi and logistic regression layer using the feature amount FCi from thememory 90. Then, with Expression (2), the one-classification determiningunit 12 calculates a calculation result y of the case where a matrixincluding the feature amount FAi and the feature amount FCi is regardedas f ((a)), a calculation result ya of the case where a matrix onlyincluding the feature amount FAi is regarded as f ((b)), and acalculation result yc of the case where a matrix only including thefeature amount FCi is regarded as f ((c)).

(iv) Step S1004

The one-classification determining unit 12 compares the calculatedcalculation result y and a threshold Th1 to each other. Specifically,when calculation result y≥threshold Th1, the processing proceeds to Step1005. When calculation result y<threshold Th1, on the other hand, theprocessing proceeds to Step 1006.

(v) Step S1005

The one-classification determining unit 12 sets the abnormal tissue orabnormal cell (for example, 1) to a classification result res.

(vi) Step S1006

The one-classification determining unit 12 sets the normal tissue ornormal cell (for example, 0) to the classification result res.

(vii) Step S1007

The one-classification determining unit 12 makes a lesion probabilityclassification from the classification result res. For example, withregard to the prostate, a result such as non-tumor or tumor is set tothe classification result res. Thus, from the classification result res,the presence or absence of a lesion (for example, tumor) or lesionprobability (y=0.89: range (0 to 1)) can be obtained. Further, theone-classification determining unit 12 can obtain lesion probability(ya=0.76: range (0 to 1)) calculated with the use of the feature amountFAi obtained from the input image, and lesion probability (yc=0.81:range (0 to 1)) calculated with the use of the feature amount FCiestimated from the input image.

(viii) Step S1008

The drawing unit 13 draws, in a case where the tissues or cells havebeen classified as abnormal, a detection frame indicating abnormaltissues or abnormal cells on the image to be displayed as illustrated inFIG. 8 . The drawing unit 13 draws no detection frame on the image in acase where the tissues or cells have been classified as normal. Further,the drawing unit displays, as illustrated in FIG. 11 , the values oflesion probability calculated from the input image and lesionprobability estimated from the input image.

(ix) Step S1009

The recording unit 14 saves, in the memory 90 (corresponding to storageapparatus 203), coordinate information with which the drawing unit 13draws a detection frame on the target image input to the drawing unit13, and the target image.

According to the first embodiment, the discriminator (including eachfeature extractor and logistic regression layer) that classifies thetissues or cells into normal and abnormal is created, by machinelearning, from an input image, a feature amount of tissues or cells inthe input image and a feature amount of tissues or cells in an imagehaving a component different from that of the input image to calculate aweight, a filter factor, and an offset. This prevents false detection orover-detection of a lesion, and makes it possible to classify, from animage, tissues or cells into normal tissues, abnormal tissues, normalcells, and abnormal cells.

Further, from an input image, a feature amount of tissues or cells in animage having a component different from that of the input image isestimated, and hence lesion probability that cannot be determined fromthe input image alone can be determined. Further, from an input image, afeature amount of tissues or cells in an image having a componentdifferent from that of the input image is estimated, and hence themanufacturing cost of the image having a component different from thatof the input image is cut, which can lead to a reduction in inspectioncost.

(2) Second Embodiment

FIG. 12 is a diagram illustrating a configuration example of an imagediagnosis assisting apparatus 2 according to a second embodiment. Theimage diagnosis assisting apparatus 2 according to the second embodimentincludes many configurations similar to those of the image diagnosisassisting apparatus 1 (see FIG. 1 ) according to the first embodiment.In the second embodiment, however, the feature extracting unit 11, theone-classification determining unit 12, and the learning unit 15 operatedifferently from those in the mode illustrated in FIG. 1 . Further, theimage diagnosis assisting apparatus 2 according to the second embodimentincludes the image generating unit 20 as a new configuration. Theconfigurations different from those in FIG. 1 are thus mainly describedhere.

The image diagnosis assisting apparatus 2 according to the secondembodiment generates, from an input image, an image stained by anotherstaining method different from a staining method for the input image.The image diagnosis assisting apparatus 2 then calculates featureamounts of tissues or cells in the input image and the generated imageto determine lesion probability of the tissues or cells in the inputimage by using these feature amounts.

<Configuration and Operation of Each Unit>

Now, the configuration and operation of each element that are differentfrom those in FIG. 1 are described in detail.

(i) Image Generating Unit 20

The image generating unit 20 uses, as illustrated in FIG. 13 , an imagegenerator D created by a learning unit 25, which is described later, togenerate, from the input image A1, an image D1 having a componentdifferent from that of the input image, and outputs the input image andthe generated image to a feature extracting unit 21.

(ii) Feature Extracting Unit 21

The feature extracting unit 21 inputs the input image A1 to the featureextractor A illustrated in FIG. 6 to calculate the feature amount FAi,and inputs, instead of the image B1, the generated image D1 to thefeature extractor B illustrated in FIG. 6 to calculate a feature amountFDi.

(iii) One-Classification Determining Unit 22

A one-classification determining unit 22 uses a matrix f of the featureamount FAi of the feature extractor A and the feature amount FDi of thefeature extractor B obtained by the feature extracting unit 21 tocalculate a value of lesion probability by logistic regression withExpression (2), to thereby determine whether the tissues or cells in theinput image A1 are normal or abnormal.

(iv) Learning Unit 25

With Expression (1) and Expression (2), the learning unit 25 learns afeature amount of an image by using, for example, a well-known machinelearning technology so that, from an input image, an image having acomponent different from that of the input image is generated. As themachine learning technology, for example, autoencoders may be used.

As illustrated in FIG. 13 , through prior machine learning, the learningunit 25 creates the image generator D configured to generate, from theinput image A1 (for example, HE stained image), the image D1 having acomponent different from that of the input image (for example,immunostained image or image subjected to special stains).

Further, as illustrated in FIG. 6 , the learning unit 25 creates thefeature extractor A and the feature extractor B, like the learning unit15. The learning unit 25 thus calculates the weight w, the filter factorwj, and the offset values b and bi of each of the feature extractors Aand B, and the filter factor wj and the offset value bi of the imagegenerator D, and stores the values in the memory.

<Hardware Configuration of Image Diagnosis Assisting Apparatus>

The image diagnosis assisting apparatus 2 according to the secondembodiment has a configuration similar to that in FIG. 2 . In the secondembodiment, however, the memory 202 includes the image generating unit20 unlike the image diagnosis assisting apparatus 1 according to thefirst embodiment.

The storage apparatus 203 of the image diagnosis assisting apparatus 2stores, for example, processing target images, a classification resultof an input image generated by the one-classification determining unit22 and a numerical value thereof, an image generated by the imagegenerating unit 20 to have a component different from that of the inputimage, positional information for drawing a detection frame generated bythe drawing unit 13, and each parameter of Expression (1) and Expression(2) generated by the learning unit 25.

FIG. 14 is a flow chart illustrating the operation of the imagediagnosis assisting apparatus 2 according to the present embodiment. Inthe following description, each processing unit (input unit 10, featureextracting unit 21, or another unit) is regarded as an operationsubject, but the description may be read as having, as the operationsubject, the CPU 201 configured to execute each processing unit servingas the program.

(i) Step 1401

The input unit 10 outputs the input image A1 to the image generatingunit 20.

(ii) Step 1402

The image generating unit 20 generates, from the input image A1, theimage D1 having a component different from that of the input image, byusing the image generator D.

(iii) Step 1403

The feature extracting unit 21 reads the filter factor wj and the offsetbi of each of the feature extractors A and B from the memory 90. Then,with Expression (1) described above, the feature extracting unit 21obtains, with the use of the filters, the feature amount FAi of thetissues or cells in the input image A1 and the feature amount FDi oftissues or cells in the input image D1.

(iv) Step 1404

The one-classification determining unit 22 reads the weight w and theoffset b of each of logistic regression layer using the feature amountFAi and logistic regression layer using the feature amount FDi from thememory 90. Then, with Expression (2), the one-classification determiningunit 22 calculates the calculation result y of a case where a matrixincluding the feature amount FAi and the feature amount FDi is regardedas f ((a1)), the calculation result ya of a case where a matrix onlyincluding the feature amount FAi is regarded as f ((b1)), and thecalculation result yc of a case where a matrix only including thefeature amount FDi is regarded as f ((c1)).

(v) Step 1405

The one-classification determining unit 22 compares the calculatedcalculation result y and the threshold Th1 to each other. Specifically,when calculation result y≥threshold Th1, the processing proceeds to Step1406. When calculation result y<threshold Th1, on the other hand, theprocessing proceeds to Step 1407.

(vi) Step 1406

The one-classification determining unit 22 sets the abnormal tissue orabnormal cell (for example, 1) to the classification result res.

(vii) Step 1407

The one-classification determining unit 22 sets the normal tissue ornormal cell (for example, 0) to the classification result res.

(viii) Step 1408

The one-classification determining unit 22 makes a lesion probabilityclassification from the classification result res. For example, withregard to the prostate, a result such as non-tumor or tumor is set tothe classification result res. Thus, from the classification result res,the presence or absence of a lesion (for example, tumor) or lesionprobability (y=0.89: range (0 to 1)) can be obtained. Further, theone-classification determining unit 22 can obtain lesion probability(ya=0.76: range (0 to 1)) calculated with the use of the feature amountFAi obtained from the input image, and lesion probability (yc=0.80:range (0 to 1)) calculated with the use of the feature amount FDi of theimage D1 generated from the input image.

(ix) Step 1409

The drawing unit 13 draws, in a case where the tissues or cells havebeen classified as abnormal, a detection frame indicating abnormaltissues or abnormal cells on the image to be displayed as illustrated inFIG. 8 . The drawing unit 13 draws no detection frame on the image in acase where the tissues or cells have been classified as normal.

Meanwhile, the drawing unit 13 displays, as illustrated in FIG. 11A, forexample, a value of lesion probability calculated from an input imagethat is a 10× image and a value of lesion probability calculated from agenerated image. As illustrated in FIG. 11B, the drawing unit 13displays, for example, a value of lesion probability calculated from aninput image that is a 40× image and a value of lesion probabilitycalculated from a generated image. That is, the drawing unit 13 displaysa plurality of determination results depending on magnifications to makeit possible to determine lesion probability on the basis of the resultsin the respective magnifications, and displays an item exceeding athreshold. With this, the results in different image magnifications arecompared to each other so that more accurate lesion probability can bedetermined. As illustrated in FIG. 11C, for example, the drawing unit 13uses the lesion probability determination results in the respectivemagnifications to display a comprehensive lesion probabilitydetermination result (for example, poorly differentiated tubularadenocarcinoma and moderately differentiated tubular adenocarcinoma).

(x) Step 1410

The recording unit 14 saves, in the memory 90 (corresponding to storageapparatus 203), coordinate information with which the drawing unit 13draws a detection frame on the target image input to the drawing unit13, and the target image.

According to the second embodiment as described above, the discriminator(including each feature extractor and logistic regression layer) thatclassifies the tissues or cells into normal and abnormal is created bymachine learning, from an input image, a feature amount of tissues orcells in the input image and a feature amount of tissues or cells in animage having a component different from that of the input image tocalculate a weight, a filter factor, and an offset. This prevents falsedetection or over-detection of a lesion, and makes it possible toclassify, from an image, tissues or cells into normal tissues, abnormaltissues, normal cells, and abnormal cells.

Further, from an input image, an image having a component different fromthat of the input image is generated, and a feature amount of tissues orcells in the image is calculated with the use of the input image and thegenerated image, and hence, lesion probability that cannot be determinedfrom the input image alone can be determined.

Further, an image having a component different from that of an inputimage is generated from the input image, and hence the manufacturingcost of the image having a component different from that of the inputimage is cut, which can lead to a reduction in inspection cost.

(3) Third Embodiment

FIG. 15 is a functional block diagram illustrating the configuration ofa remote diagnosis assisting system 1500 according to a thirdembodiment. The remote diagnosis assisting system 1500 includes a serveror the like 1503 and an image acquiring apparatus 1505.

The image acquiring apparatus 1505 is an apparatus, such as a virtualslide apparatus or a personal computer equipped with a camera, andincludes an imaging unit 1501 configured to capture image data and adisplay unit 1504 configured to display a determination resulttransmitted from the server or the like 1503. Note that, the imageacquiring apparatus 1505 includes, although not illustrated, acommunication device configured to send image data to the server or thelike 1503 and receive data sent from the server or the like 1503.

The server or the like 1503 includes the image diagnosis assistingapparatus 1 configured to perform, on image data transmitted from theimage acquiring apparatus 1505, the image processing according to thefirst or second embodiment of the present invention, and a storage unit1502 configured to store a determination result output from the imagediagnosis assisting apparatus 1. Note that, the server or the like 1503includes, although not illustrated, a communication device configured toreceive image data sent from the image acquiring apparatus 1505 and senddetermination result data to the image acquiring apparatus 1505.

The image diagnosis assisting apparatus 1 makes a classification ontissues or cells in image data captured by the imaging unit 1501 todetermine the presence or absence of abnormal tissues or abnormal cells(for example, cancer). Further, the image diagnosis assisting apparatus1 uses a result of classification by a discriminator configured tocalculate a feature amount of tissues or cells in an input image and afeature amount of tissues or cells in an image having a componentdifferent from that of the input image, to thereby make a classificationon lesion probability of abnormal tissues or abnormal cells (forexample, cancer) depending on the progression of abnormal tissues orabnormal cells (for example, cancer). The display unit 1504 displays aclassification result transmitted from the server or the like 1503 onthe display screen of the image acquiring apparatus 1505.

Examples of the image acquiring apparatus 1505 may include apparatus forregenerative medicine or iPS cell culture apparatus including an imagecapturing unit, MRI, and ultrasonic image capturing apparatus.

According to the third embodiment, tissues or cells in an imagetransmitted from a facility or the like at a different location areclassified into normal tissues, abnormal tissues, normal cells, andabnormal cells, and the classification result is transmitted to thefacility or the like at a different location so that a display unit ofan image acquiring apparatus in the facility or the like displays theclassification result. The remote diagnosis assisting system cantherefore be provided.

(4) Fourth Embodiment

FIG. 16 is a functional block diagram illustrating the configuration ofan online contract service providing system 1600 according to a fourthembodiment of the present invention. The online contract serviceproviding system 1600 includes a server or the like 1603 and an imageacquiring apparatus 1605.

The image acquiring apparatus 1605 is an apparatus, such as a virtualslide apparatus or a personal computer equipped with a camera, andincludes a imaging unit 1601 configured to capture image data, a storageunit 1604 configured to store a discriminator transmitted from theserver or the like 1603, and the image diagnosis assisting apparatus 1configured to perform the image processing according to the first andsecond embodiments, that is, to read a discriminator transmitted fromthe server or the like 1603, thereby classifying tissues or cells in animage newly captured by the imaging unit 1601 of the image acquiringapparatus 1605 into normal tissues, abnormal tissues, normal cells, andabnormal cells.

Note that, the image acquiring apparatus 1605 includes, although notillustrated, a communication device configured to send image data to theserver or the like 1603 and receive data sent from the server or thelike 1603.

The server or the like 1603 includes the image diagnosis assistingapparatus 1 configured to perform, on image data transmitted from theimage acquiring apparatus 1605, the image processing according to thefirst or second embodiment of the present invention, and a storage unit1602 configured to store a discriminator output from the image diagnosisassisting apparatus 1. Note that, the server or the like 1603 includes,although not illustrated, a communication device configured to receiveimage data sent from the image acquiring apparatus 1605 and send adiscriminator to the image acquiring apparatus 1605.

The image diagnosis assisting apparatus 1 performs machine learning todetermine, with regard to tissues or cells in image data captured by theimaging unit 1601, normal tissues or cells as normal tissues or cellsand abnormal tissues or cells as abnormal tissues or cells, to therebycreate a discriminator configured to calculate a feature amount oftissues or cells in an image at a facility or the like at a differentlocation and a feature amount of tissues or cells in an image having acomponent different from that of the image.

The storage unit 1604 stores a discriminator or the like transmittedfrom the server or the like 1603.

The image diagnosis assisting apparatus 1 in the image acquiringapparatus 1605 reads a discriminator or the like from the storage unit1604, and classifies, by using the discriminator, tissues or cells in animage newly captured by the imaging unit 1601 of the image acquiringapparatus 1605 into normal tissues, abnormal tissues, normal cells, andabnormal cells. The image diagnosis assisting apparatus 1 displays theclassification result on the display screen of the output apparatus 204thereof.

Examples of the image acquiring apparatus 1605 may include apparatus forregenerative medicine or iPS cell culture apparatus including an imagecapturing unit, MRI, and ultrasonic image capturing apparatus.

According to the fourth embodiment, a discriminator or the like iscreated by performing machine learning so that, with regard to tissuesor cells in an image transmitted from a facility or the like at adifferent location, normal tissues or cells are classified as normaltissues or cells and abnormal tissues or cells are classified asabnormal tissues or cells, and the discriminator or the like istransmitted to the facility or the like at a different location so thatan image acquiring apparatus in the facility or the like reads thediscriminator to classify tissues or cells in a newly captured imageinto normal tissues, abnormal tissues, normal cells, and abnormal cells.The online contract service providing system can therefore be provided.

In each embodiment described above, the following modifications can bemade. For example, the feature extracting units 11 and 21 and thelearning units 15 and 25, which obtain a plurality of feature amounts byusing the filters through machine learning, may use another featureamount such as HOG. A similar effect is provided also in this case.

The one-classification determining units 12 and 22, which obtain afeature amount of tissues or cells by using logistic regression throughmachine learning, may use linear regression or Poisson regression, forexample. A similar effect is provided also in this case.

The feature extracting unit 11 and the feature extracting unit 21, whichcalculate a feature amount of an input image or feature amounts of aninput image and a generated image by using the two feature extractors,may calculate a feature amount by using one feature extractor or threeor more feature extractors. A similar effect is provided also in thiscase.

The present invention can also be implemented by a program code ofsoftware that implements the functions of the embodiments. In this case,a storage medium having the program code recorded thereon is provided toa system or an apparatus, and the computer (or CPU or MPU) of the systemor the apparatus reads the program code stored in the storage medium. Inthis case, the program code itself read from the storage mediumimplements the functions of the embodiments described above, and theprogram code itself and the storage medium having the program codestored therein configure the present invention. Examples of the storagemedium for supplying the program code include flexible disks, CD-ROMs,DVD-ROMs, hard disks, optical discs, magneto-optical discs, CD-Rs,magnetic tapes, nonvolatile memory cards, and ROMs.

Further, an operating system (OS) running on the computer, for example,may perform a part or the entire of the practical processing on thebasis of an instruction of the program code, thereby implementing thefunctions of the embodiments described above by the processing. Inaddition, for example, the CPU of the computer may perform, after theprogram code read from the storage medium is written in a memory on thecomputer, a part or the entire of the practical processing on the basisof an instruction of the program code, thereby implementing thefunctions of the embodiments described above by the processing.

In addition, the program code of the software that implements thefunctions of the embodiments may be delivered via the network to bestored in storage means in the system or the apparatus, such as a harddisk or a memory, or the storage medium, such as CD-RW or CD-R, so thatthe computer (or CPU or MPU) of the system or the apparatus in use mayread and execute the program code stored in the storage means or thestorage medium.

Finally, the processes and the technology described herein areessentially not related to any specific apparatus, and can also beimplemented by any suitable combination of the components. In addition,various general-purpose devices can be used according to the methoddescribed herein. In executing the steps of the method described herein,building a dedicated apparatus is sometimes advantageous. Further,appropriate combinations of the plurality of components disclosed in theembodiments make it possible to form various inventions. For example,several components may be removed from all the components described inthe embodiments. In addition, the components of the differentembodiments may be appropriately combined with each other. The presentinvention is described in association with the specific examples, butthe specific examples are not intended to impose any limitation but tofacilitate the description. Persons who have ordinary knowledge in theart definitely understand that there are a large number of suitablecombinations of hardware, software, and firmware in implementing thepresent invention. For example, the above-mentioned software can beimplemented by a wide range of programs or script languages, such asAssembler, C/C++, Perl, Shell, PHP, and Java (registered trademark).

Furthermore, in the above-mentioned embodiments, control lines andinformation lines that are considered to be necessary for thedescription are described, and the control lines or the informationlines do not necessarily indicate all control lines or information linesof a product. All the configurations may be connected to each other.

In addition, other implementation forms of the present invention areapparent for persons who have ordinary knowledge in the art byconsidering the specification and the embodiments of the presentinvention disclosed herein. Various aspects and/or components of thedescribed embodiments can be used independently or can be combined inany manner.

DESCRIPTION OF REFERENCE CHARACTERS

-   1: Image diagnosis assisting apparatus-   10: Input unit-   11: Feature extracting unit-   12: One-classification determining unit-   13: Drawing unit-   14: Recording unit-   15: Learning unit-   20: Image generating unit-   21: Feature extracting unit-   22: One-classification determining unit-   25: Learning unit-   91: Control unit-   1500: Remote diagnosis assisting system-   1600: Online contract service providing system

The invention claimed is:
 1. An image diagnosis assisting apparatus,comprising: a processor configured to execute various programs forperforming image processing on a target image; and a memory configuredto store a result of the image processing, wherein the processorexecutes: processing of inputting an image of tissues or cells;processing of extracting feature amounts of tissues or cells in thetarget image; feature estimation processing of estimating featureamounts of an image dyed by another dyeing method different from thedyeing method of the target image by machine learning from image data oftissues or cells in the target image; and determination processing ofdetermining presence or absence of a lesion and lesion probability foreach of the target images by using a plurality of feature amountsincluding at least the feature amounts of tissues or cells in the targetimage and the feature amounts of an image dyed by another dyeing method.2. The image diagnosis assisting apparatus according to claim 1, whereinin the feature estimation processing, the processor estimates thefeature amounts of the image dyed by another dyeing method differentfrom the dyeing method of the target image by machine learning from theimage data of the tissues or cells in the target image throughestimation based on the target image.
 3. The image diagnosis assistingapparatus according to claim 1, wherein in the determination processing,the processor determines the presence or absence of the lesion and thelesion probability by using a discriminator configured to calculate fromthe feature amounts of the tissues or cells in the target image, thefeature amounts of the image dyed by another dyeing method differentfrom the dyeing method of the target image by machine learning.
 4. Theimage diagnosis assisting apparatus according to claim 1, wherein theprocessor displays a plurality of determination results depending onmagnifications to determine the lesion probability based on the resultsin the respective magnifications.
 5. An image diagnosis assistingapparatus, comprising: a processor configured to execute variousprograms for performing image processing on a target image; and a memoryconfigured to store a result of the image processing, wherein theprocessor executes processing of inputting an image of tissues or cells,processing of extracting feature amounts of tissues or cells in thetarget image, processing of generating, from the target image, an imagedyed by another dyeing method different from the dyeing method of thetarget image by machine learning, feature estimation processing ofestimating feature amounts of tissues or cells in the generated image,and determination processing of determining presence or absence of alesion and lesion probability for each of the target images by using aplurality of feature amounts including at least the feature amounts oftissues or cells in the target image and the feature amounts of an imagedyed by another dyeing method.
 6. The image diagnosis assistingapparatus according to claim 5, wherein the processor displays aplurality of determination results depending on magnifications todetermine the lesion probability based on the results in the respectivemagnifications.
 7. An image diagnosis assisting apparatus, comprising: aprocessor configured to execute various programs for performing imageprocessing on a target image; and a memory configured to store a resultof the image processing, wherein the processor executes processing ofinputting an image of tissues or cells, processing of extracting featureamounts of tissues or cells in the target image, processing ofgenerating, from the target image, an image dyed by another dyeingmethod different from the dyeing method of the target image by machinelearning, feature estimation processing of estimating feature amounts oftissues or cells in the generated image, and determination processing ofdetermining presence or absence of a lesion and lesion probability foreach of the target images by using the feature amounts extracted by theprocessing of extracting and the feature amounts estimated by thefeature estimation processing, wherein in the determination processing,the processor determines the presence or absence of a lesion and thelesion probability by using a discriminator configured to calculate,from the feature amounts of the tissues or cells in the image, thefeature amounts of the image dyed by another dyeing method differentfrom the dyeing method of the target image by machine learning.
 8. Animage diagnosis assisting method for classifying desired tissues orcells in a target image, comprising: inputting an image of tissues orcells by a processor configured to execute various programs forperforming image processing on the target image; extracting, by theprocessor, feature amounts of tissues or cells in the target image;feature estimating, by the processor, feature amounts of an image dyedby another dyeing method different from the dyeing method of the targetimage by machine learning from the image data of tissues or cells in thetarget image; and determining, by the processor, presence or absence ofa lesion and lesion probability for each of the target images by using aplurality of feature amounts including at least the feature amounts oftissues or cells in the target image and the feature amounts of an imagedyed by another dyeing method.
 9. The image diagnosis assisting methodaccording to claim 8, wherein in the feature estimating, the processorestimates the feature amounts of the image dyed by another dyeing methoddifferent from the dyeing method of the target image by machine learningfrom the image data of the tissues or cells in the target image throughestimation based on the target image.
 10. The image diagnosis assistingmethod according to claim 8, wherein the processor determines thepresence or absence of the lesion and the lesion probability by using adiscriminator configured to calculate from the feature amounts of thetissues or cells in the target image, the feature amounts of the imagedyed by another dyeing method different from the dyeing method of thetarget image by machine learning.
 11. The image diagnosis assistingmethod according to claim 8, wherein the processor displays a pluralityof determination results depending on magnifications to determine thelesion probability based on the results in the respectivemagnifications.
 12. An image diagnosis assisting method for classifyinga desired tissues or cells in a target image, comprising: inputting animage of tissues or cells by a processor configured to execute variousprograms for performing image processing on the target image;extracting, by the processor, feature amounts of tissues or cells in thetarget image; generating, by the processor, from the target image, animage dyed by another dyeing method different from the dyeing method ofthe target image by machine learning; feature estimating, by theprocessor, feature amounts of tissues or cells in the generated image;and determining, by the processor, presence or absence of a lesion andlesion probability for each of the target images by using a plurality ofthe feature amounts including at least the feature amounts of tissues orcells in the target image and the feature amounts of an image dyed byanother dyeing method.
 13. An image diagnosis assisting method forclassifying desired tissues or cells in a target image, comprising:inputting an image of tissues or cells by a processor configured toexecute various programs for performing image processing on the targetimage; extracting, by the processor, feature amounts of tissues or cellsin the target image; generating, by the processor, from the targetimage, an image dyed by another dyeing method different from the dyeingmethod of the target image by machine learning; feature estimating, bythe processor, feature amounts of tissues or cells in the generatedimage; and determining, by the processor, presence or absence of alesion and lesion probability for each of the target images by using aplurality of the feature amounts, wherein in the determining, theprocessor determines the presence or absence of a lesion and the lesionprobability by using a discriminator configured to calculate, from thefeature amounts of the tissues or cells in the target image, the featureamounts of the image dyed by another dyeing method different from thedyeing method of the target image by machine learning.
 14. A remotediagnosis assisting system, comprising: a server including an imagediagnosis assisting apparatus, the image diagnosis assisting apparatusincluding a processor configured to execute various programs forperforming image processing on a target image, and a memory configuredto store a result of the image processing, the processor executingprocessing of inputting an image of tissues or cells, processing ofextracting feature amounts of tissues or cells in the target image,feature estimation processing of estimating feature amounts of an imagedyed by another dyeing method different from the dyeing method of thetarget image by machine learning from the image data of tissues or cellsin the target image, or of generating, from the target image, an imagedyed by another dyeing method different from the dyeing method of thetarget image by machine learning and extracting feature amounts oftissues or cells in the generated image, and determination processing ofdetermining presence or absence of a lesion and lesion probability foreach of the target images by setting a classification based on theresult and using the feature amounts extracted by the processing ofextracting and the feature amounts estimated by the feature estimationprocessing; and an image acquiring apparatus including an imagingapparatus configured to capture image data, wherein the image acquiringapparatus sends the image data to the server, the server processes, bythe image diagnosis assisting apparatus, the image data that the serverhas received, and stores, in the memory, the image of the tissues orcells on which the determination has been made and a result of thedetermination and sends the image of the tissues or cells on which thedetermination has been made and the result of the determination to theimage acquiring apparatus, and the image acquiring apparatus displays,on a display apparatus, the image of the tissues or cells on which thedetermination has been made and the result of the determination that theimage acquiring apparatus has received.
 15. An online contract serviceproviding system, comprising: a server including an image diagnosisassisting apparatus, the image diagnosis assisting apparatus including aprocessor configured to execute various programs for performing imageprocessing on a target image, and a memory configured to store a resultof the image processing, the processor executing processing of inputtingan image of tissues or cells, processing of extracting feature amountsof tissues or cells in the target image, feature estimation processingof estimating feature amounts of an image dyed by another dyeing methoddifferent from the dyeing method of the target image by machine learningfrom the image data of tissues or cells in the target image, or ofgenerating, from the target image, an image dyed by another dyeingmethod different from the dyeing method of the target image by machinelearning and extracting feature amounts of tissues or cells in thegenerated image, and determination processing of determining presence orabsence of a lesion and lesion probability for each of the target imagesby setting a classification based on the result and using the featureamounts extracted by the processing of extracting and the featureamounts estimated by the feature estimation processing; and an imageacquiring apparatus including an imaging apparatus configured to capturethe image data, and the image diagnosis assisting apparatus, wherein theimage acquiring apparatus sends the image data to the server, the serverprocesses, by the image diagnosis assisting apparatus, the image datathat the server has received, and stores, in the memory, the image ofthe tissues or cells on which the determination has been made and adiscriminator and sends the image of the tissues or cells on which thedetermination has been made and the discriminator to the image acquiringapparatus, the image acquiring apparatus stores the image of the tissuesor cells on which the determination has been made and the discriminatorthat the image acquiring apparatus has received, and the image diagnosisassisting apparatus in the image acquiring apparatus makes adetermination on an image of another tissues or cells by using thediscriminator, and displays a result of the determination on a displayapparatus.